155 research outputs found
Reinforcement Learning with Perturbed Rewards
Recent studies have shown that reinforcement learning (RL) models are
vulnerable in various noisy scenarios. For instance, the observed reward
channel is often subject to noise in practice (e.g., when rewards are collected
through sensors), and is therefore not credible. In addition, for applications
such as robotics, a deep reinforcement learning (DRL) algorithm can be
manipulated to produce arbitrary errors by receiving corrupted rewards. In this
paper, we consider noisy RL problems with perturbed rewards, which can be
approximated with a confusion matrix. We develop a robust RL framework that
enables agents to learn in noisy environments where only perturbed rewards are
observed. Our solution framework builds on existing RL/DRL algorithms and
firstly addresses the biased noisy reward setting without any assumptions on
the true distribution (e.g., zero-mean Gaussian noise as made in previous
works). The core ideas of our solution include estimating a reward confusion
matrix and defining a set of unbiased surrogate rewards. We prove the
convergence and sample complexity of our approach. Extensive experiments on
different DRL platforms show that trained policies based on our estimated
surrogate reward can achieve higher expected rewards, and converge faster than
existing baselines. For instance, the state-of-the-art PPO algorithm is able to
obtain 84.6% and 80.8% improvements on average score for five Atari games, with
error rates as 10% and 30% respectively.Comment: AAAI 2020 (Spotlight
UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generation
LiDAR provides accurate geometric measurements of the 3D world.
Unfortunately, dense LiDARs are very expensive and the point clouds captured by
low-beam LiDAR are often sparse. To address these issues, we present
UltraLiDAR, a data-driven framework for scene-level LiDAR completion, LiDAR
generation, and LiDAR manipulation. The crux of UltraLiDAR is a compact,
discrete representation that encodes the point cloud's geometric structure, is
robust to noise, and is easy to manipulate. We show that by aligning the
representation of a sparse point cloud to that of a dense point cloud, we can
densify the sparse point clouds as if they were captured by a real high-density
LiDAR, drastically reducing the cost. Furthermore, by learning a prior over the
discrete codebook, we can generate diverse, realistic LiDAR point clouds for
self-driving. We evaluate the effectiveness of UltraLiDAR on sparse-to-dense
LiDAR completion and LiDAR generation. Experiments show that densifying
real-world point clouds with our approach can significantly improve the
performance of downstream perception systems. Compared to prior art on LiDAR
generation, our approach generates much more realistic point clouds. According
to A/B test, over 98.5\% of the time human participants prefer our results over
those of previous methods.Comment: CVPR 2023. Project page: https://waabi.ai/ultralidar
Adv3D: Generating Safety-Critical 3D Objects through Closed-Loop Simulation
Self-driving vehicles (SDVs) must be rigorously tested on a wide range of
scenarios to ensure safe deployment. The industry typically relies on
closed-loop simulation to evaluate how the SDV interacts on a corpus of
synthetic and real scenarios and verify it performs properly. However, they
primarily only test the system's motion planning module, and only consider
behavior variations. It is key to evaluate the full autonomy system in
closed-loop, and to understand how variations in sensor data based on scene
appearance, such as the shape of actors, affect system performance. In this
paper, we propose a framework, Adv3D, that takes real world scenarios and
performs closed-loop sensor simulation to evaluate autonomy performance, and
finds vehicle shapes that make the scenario more challenging, resulting in
autonomy failures and uncomfortable SDV maneuvers. Unlike prior works that add
contrived adversarial shapes to vehicle roof-tops or roadside to harm
perception only, we optimize a low-dimensional shape representation to modify
the vehicle shape itself in a realistic manner to degrade autonomy performance
(e.g., perception, prediction, and motion planning). Moreover, we find that the
shape variations found with Adv3D optimized in closed-loop are much more
effective than those in open-loop, demonstrating the importance of finding
scene appearance variations that affect autonomy in the interactive setting.Comment: CoRL 2023. Project page: https://waabi.ai/adv3d
Recent Progress on Nanostructures for Drug Delivery Applications
With the rapid development of nanotechnology, the convergence of nanostructures and drug delivery has become a research hotspot in recent years. Due to their unique and superior properties, various nanostructures, especially those fabricated from self-assembly, are able to significantly increase the solubility of poorly soluble drugs, reduce cytotoxicity toward normal tissues, and improve therapeutic efficacy. Nanostructures have been successfully applied in the delivery of diverse drugs, such as small molecules, peptides, proteins, and nucleic acids. In this paper, the driving forces for the self-assembly of nanostructures are introduced. The strategies of drug delivery by nanostructures are briefly discussed. Furthermore, the emphasis is put on a variety of nanostructures fabricated from various building materials, mainly liposomes, polymers, ceramics, metal, peptides, nucleic acids, and even drugs themselves
- …